Eigen-points: Control-point Location using Principle Component Analyses

Eigen-points estimates the image-plane locations of fiduciary points on an objects. By estimating multiple locations simultaneously, eigen-points exploits the interdependence between these locations. This is done by associating neighboring, inter-dependent control-points with a model of the local appearance. The model of local appearance is used to find the feature in new unlabeled images. Control-point locations are then estimated from the appearance of this feature in the unlabeled image. The estimation is done using an affine manifold model of the coupling between the local appearance and the local shape. Eigen-points uses models aimed specifically at recovering shape from image appearance. The estimation equations are solved non-iteratively, in a way that accounts for noise in the training data and the unlabeled images and that accounts for uncertainty in the distribution and dependencies within these noise sources. original original original automatic morph automatic morph Figure 1: Examples of image morphs using automatically placed correspondences The control-point locations for these morphs were estimated automatically by eigen-points. Constraints were placed around eyes, nose, mouth, chin and ears. No constraints were placed on the hair. Interval Research Corporation Technical Report # 1996-060 Copyright 1996 IEEE. Published in the Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, Oct 14-16, 1996. Killington, VT. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE. Contact: Manager, Copyrights and Permissions / IEEE Service Center / 445 Hoes Lane / P.O. Box 1331 / Piscataway, NJ 08855-1331, USA. Telephone: + Intl. 908-562-3966. them to take advantage of example-based learning to constrain the estimated locations of these control points. However, there is no direct link between the image appearance (the external-energy term) and the shape constraints (the internal-energy term). This makes the discovery of “correct” energy functional an error-prone process. Shape-plus-texture models [8][9] describe appearance using two separate reconstructive models: one for shape (e.g. contour locations) and one for shape-free texture. The shape-free texture descriptions model the grayscale values under object-centric sampling. Thus, the texture models do not describe the observed grayscale data, but instead describe the grayscale data resampled according to the estimated shape description. These shape-plus-texture approaches give simultaneous estimates for many controlpoint locations. They have well-defined example-based training methods and an error criteria derived from training. However, the texture models use an estimate of shape. Thus, they are forced to rely on iterative solutions to find consistent shape and texture estimates. Another drawback to shape-plus-texture approaches is their use of reconstructive as opposed to discriminative models. The texture model capture the principal variations of the (shape-normalized) appearance, giving the minimum mean-square error reconstruction for a given description length. However, our goal is to find a good estimate for the true shape, not to find a good estimate for the true appearance (shape-normalized or otherwise). Instead of a reconstructive texture model, we need a “shape-discriminant” model of texture. That is, we need the model that best captures the principal variations of shape, as manifested in appearance. The next section describes our approach to discriminating between shapes based on the observed image data. 3 Eigen-point approach to placing control points Using eigen-points, the problem of locating fiduciary points on an unmarked image is solved in two stages. First, the location of features * are estimated; then control points are placed around that feature. The first stage locates the feature of interest—for example, the actors’ lips. This can be done using templateor model-based matching. The feature location defines both the subimage and the image-plane origin that are used in the second stage. The second stage places the control points around the feature—for example, marking the locations in the image that show the outer boundary of the lips. The locations of the fiduciary points are estimated using an affine manifold model that couples the grayscale values within the feature to the control-point locations associated with the feature. This approach effectively assumes that there is a single K dimensional vector, , which drives both the feature grayscale vector and the control-point locations. The functions which transform this vector into appearance and shape are assumed to be affine. Assuming a coupled, affine model for image-plane shape and appearance, the defining equations for the grayscale values and the control-point locations are: